{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "xLOXFOT5Q40E"
      },
      "source": [
        "##### Copyright 2020 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "iiQkM5ZgQ8r2"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "j6331ZSsQGY3"
      },
      "source": [
        "# Calculate gradients"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "i9Jcnb8bQQyd"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/quantum/tutorials/gradients\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/quantum/blob/master/docs/tutorials/gradients.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/quantum/blob/master/docs/tutorials/gradients.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/quantum/docs/tutorials/gradients.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "FxkQA6oblNqI"
      },
      "source": [
        "This tutorial explores gradient calculation algorithms for the expectation values of quantum circuits.\n",
        "\n",
        "Calculating the gradient of the expectation value of a certain observable in a quantum circuit is an involved process. Expectation values of observables do not have the luxury of having analytic gradient formulas that are always easy to write down—unlike traditional machine learning transformations such as matrix multiplication or vector addition that have analytic gradient formulas which are easy to write down. As a result, there are different quantum gradient calculation methods that come in handy for different scenarios. This tutorial compares and contrasts two different differentiation schemes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "pvG0gAJqGYJo"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "TorxE5tnkvb2"
      },
      "outputs": [],
      "source": [
        "!pip install tensorflow==2.3.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "OIbP5hklC338"
      },
      "source": [
        "Install TensorFlow Quantum:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "saFHsRDpkvkH"
      },
      "outputs": [],
      "source": [
        "!pip install tensorflow-quantum"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "MkTqyoSxGUfB"
      },
      "source": [
        "Now import TensorFlow and the module dependencies:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "enZ300Bflq80"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "import tensorflow_quantum as tfq\n",
        "\n",
        "import cirq\n",
        "import sympy\n",
        "import numpy as np\n",
        "\n",
        "# visualization tools\n",
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt\n",
        "from cirq.contrib.svg import SVGCircuit"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "b08Mmbs8lr81"
      },
      "source": [
        "## 1. Preliminary\n",
        "\n",
        "Let's make the notion of gradient calculation for quantum circuits a little more concrete. Suppose you have a parameterized circuit like this one:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "YkPYJ_Ak-GKu"
      },
      "outputs": [],
      "source": [
        "qubit = cirq.GridQubit(0, 0)\n",
        "my_circuit = cirq.Circuit(cirq.Y(qubit)**sympy.Symbol('alpha'))\n",
        "SVGCircuit(my_circuit)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "wgQIlCWy-MVr"
      },
      "source": [
        "Along with an observable:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "xurmJdFy-Jae"
      },
      "outputs": [],
      "source": [
        "pauli_x = cirq.X(qubit)\n",
        "pauli_x"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "j3OzKYe5NT_W"
      },
      "source": [
        "Looking at this operator you know that $⟨Y(\\alpha)| X | Y(\\alpha)⟩ = \\sin(\\pi \\alpha)$"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "Ps-pd2mndXs7"
      },
      "outputs": [],
      "source": [
        "def my_expectation(op, alpha):\n",
        "    \"\"\"Compute ⟨Y(alpha)| `op` | Y(alpha)⟩\"\"\"\n",
        "    params = {'alpha': alpha}\n",
        "    sim = cirq.Simulator()\n",
        "    final_state_vector = sim.simulate(my_circuit, params).final_state_vector\n",
        "    return op.expectation_from_state_vector(final_state_vector, {qubit: 0}).real\n",
        "\n",
        "\n",
        "my_alpha = 0.3\n",
        "print(\"Expectation=\", my_expectation(pauli_x, my_alpha))\n",
        "print(\"Sin Formula=\", np.sin(np.pi * my_alpha))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "zcCX109cJUaz"
      },
      "source": [
        " and if you define $f_{1}(\\alpha) = ⟨Y(\\alpha)| X | Y(\\alpha)⟩$ then $f_{1}^{'}(\\alpha) = \\pi \\cos(\\pi \\alpha)$. Let's check this:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "VMq7EayNRyQb"
      },
      "outputs": [],
      "source": [
        "def my_grad(obs, alpha, eps=0.01):\n",
        "    grad = 0\n",
        "    f_x = my_expectation(obs, alpha)\n",
        "    f_x_prime = my_expectation(obs, alpha + eps)\n",
        "    return ((f_x_prime - f_x) / eps).real\n",
        "\n",
        "\n",
        "print('Finite difference:', my_grad(pauli_x, my_alpha))\n",
        "print('Cosine formula:   ', np.pi * np.cos(np.pi * my_alpha))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "-SUlLpXBeicF"
      },
      "source": [
        "## 2. The need for a differentiator\n",
        "\n",
        "With larger circuits, you won't always be so lucky to have a formula that precisely calculates the gradients of a given quantum circuit. In the event that a simple formula isn't enough to calculate the gradient, the `tfq.differentiators.Differentiator` class allows you to define algorithms for computing the gradients of your circuits. For instance you can recreate the above example in TensorFlow Quantum (TFQ) with:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "Om76ZLu8NT_i"
      },
      "outputs": [],
      "source": [
        "expectation_calculation = tfq.layers.Expectation(\n",
        "    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))\n",
        "\n",
        "expectation_calculation(my_circuit,\n",
        "                        operators=pauli_x,\n",
        "                        symbol_names=['alpha'],\n",
        "                        symbol_values=[[my_alpha]])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "lx3y2DX9NT_k"
      },
      "source": [
        "However, if you switch to estimating expectation based on sampling (what would happen on a true device) the values can change a little bit. This means you now have an imperfect estimate:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "v27rRyAHNT_l"
      },
      "outputs": [],
      "source": [
        "sampled_expectation_calculation = tfq.layers.SampledExpectation(\n",
        "    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))\n",
        "\n",
        "sampled_expectation_calculation(my_circuit,\n",
        "                                operators=pauli_x,\n",
        "                                repetitions=500,\n",
        "                                symbol_names=['alpha'],\n",
        "                                symbol_values=[[my_alpha]])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Igwa3EnzNT_p"
      },
      "source": [
        "This can quickly compound into a serious accuracy problem when it comes to gradients:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "StljXH38NT_q"
      },
      "outputs": [],
      "source": [
        "# Make input_points = [batch_size, 1] array.\n",
        "input_points = np.linspace(0, 5, 200)[:, np.newaxis].astype(np.float32)\n",
        "exact_outputs = expectation_calculation(my_circuit,\n",
        "                                        operators=pauli_x,\n",
        "                                        symbol_names=['alpha'],\n",
        "                                        symbol_values=input_points)\n",
        "imperfect_outputs = sampled_expectation_calculation(my_circuit,\n",
        "                                                    operators=pauli_x,\n",
        "                                                    repetitions=500,\n",
        "                                                    symbol_names=['alpha'],\n",
        "                                                    symbol_values=input_points)\n",
        "plt.title('Forward Pass Values')\n",
        "plt.xlabel('$x$')\n",
        "plt.ylabel('$f(x)$')\n",
        "plt.plot(input_points, exact_outputs, label='Analytic')\n",
        "plt.plot(input_points, imperfect_outputs, label='Sampled')\n",
        "plt.legend()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "dfXObk7KNT_t"
      },
      "outputs": [],
      "source": [
        "# Gradients are a much different story.\n",
        "values_tensor = tf.convert_to_tensor(input_points)\n",
        "\n",
        "with tf.GradientTape() as g:\n",
        "    g.watch(values_tensor)\n",
        "    exact_outputs = expectation_calculation(my_circuit,\n",
        "                                            operators=pauli_x,\n",
        "                                            symbol_names=['alpha'],\n",
        "                                            symbol_values=values_tensor)\n",
        "analytic_finite_diff_gradients = g.gradient(exact_outputs, values_tensor)\n",
        "\n",
        "with tf.GradientTape() as g:\n",
        "    g.watch(values_tensor)\n",
        "    imperfect_outputs = sampled_expectation_calculation(\n",
        "        my_circuit,\n",
        "        operators=pauli_x,\n",
        "        repetitions=500,\n",
        "        symbol_names=['alpha'],\n",
        "        symbol_values=values_tensor)\n",
        "sampled_finite_diff_gradients = g.gradient(imperfect_outputs, values_tensor)\n",
        "\n",
        "plt.title('Gradient Values')\n",
        "plt.xlabel('$x$')\n",
        "plt.ylabel('$f^{\\'}(x)$')\n",
        "plt.plot(input_points, analytic_finite_diff_gradients, label='Analytic')\n",
        "plt.plot(input_points, sampled_finite_diff_gradients, label='Sampled')\n",
        "plt.legend()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Ld34TJvTNT_w"
      },
      "source": [
        "Here you can see that although the finite difference formula is fast to compute the gradients themselves in the analytical case, when it came to the sampling based methods it was far too noisy. More careful techniques must be used to ensure a good gradient can be calculated. Next you will look at a much slower technique that wouldn't be as well suited for analytical expectation gradient calculations, but does perform much better in the real-world sample based case:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "JsBxH_RaNT_x"
      },
      "outputs": [],
      "source": [
        "# A smarter differentiation scheme.\n",
        "gradient_safe_sampled_expectation = tfq.layers.SampledExpectation(\n",
        "    differentiator=tfq.differentiators.ParameterShift())\n",
        "\n",
        "with tf.GradientTape() as g:\n",
        "    g.watch(values_tensor)\n",
        "    imperfect_outputs = gradient_safe_sampled_expectation(\n",
        "        my_circuit,\n",
        "        operators=pauli_x,\n",
        "        repetitions=500,\n",
        "        symbol_names=['alpha'],\n",
        "        symbol_values=values_tensor)\n",
        "\n",
        "sampled_param_shift_gradients = g.gradient(imperfect_outputs, values_tensor)\n",
        "\n",
        "plt.title('Gradient Values')\n",
        "plt.xlabel('$x$')\n",
        "plt.ylabel('$f^{\\'}(x)$')\n",
        "plt.plot(input_points, analytic_finite_diff_gradients, label='Analytic')\n",
        "plt.plot(input_points, sampled_param_shift_gradients, label='Sampled')\n",
        "plt.legend()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "0xlUlh8wNT_z"
      },
      "source": [
        "From the above you can see that certain differentiators are best used for particular research scenarios. In general, the slower sample-based methods that are robust to device noise, etc., are great differentiators when testing or implementing algorithms in a more \"real world\" setting. Faster methods like finite difference are great for analytical calculations and you want higher throughput, but aren't yet concerned with the device viability of your algorithm."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "FaijzZ4MNT_0"
      },
      "source": [
        "## 3. Multiple observables\n",
        "\n",
        "Let's introduce a second observable and see how TensorFlow Quantum supports multiple observables for a single circuit."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "ytgB_DqDNT_3"
      },
      "outputs": [],
      "source": [
        "pauli_z = cirq.Z(qubit)\n",
        "pauli_z"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "r51TZls4NT_6"
      },
      "source": [
        "If this observable is used with the same circuit as before, then you have $f_{2}(\\alpha) = ⟨Y(\\alpha)| Z | Y(\\alpha)⟩ = \\cos(\\pi \\alpha)$ and $f_{2}^{'}(\\alpha) = -\\pi \\sin(\\pi \\alpha)$. Perform a quick check:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "19FKgu0ANT_7"
      },
      "outputs": [],
      "source": [
        "test_value = 0.\n",
        "\n",
        "print('Finite difference:', my_grad(pauli_z, test_value))\n",
        "print('Sin formula:      ', -np.pi * np.sin(np.pi * test_value))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "_33Y5mL0NT_-"
      },
      "source": [
        "It's a match (close enough).\n",
        "\n",
        "Now if you define $g(\\alpha) = f_{1}(\\alpha) + f_{2}(\\alpha)$ then $g'(\\alpha) = f_{1}^{'}(\\alpha) + f^{'}_{2}(\\alpha)$. Defining more than one observable in TensorFlow Quantum to use along with a circuit is equivalent to adding on more terms to $g$.\n",
        "\n",
        "This means that the gradient of a particular symbol in a circuit is equal to the sum of the gradients with regards to each observable for that symbol applied to that circuit. This is compatible with TensorFlow gradient taking and backpropagation (where you give the sum of the gradients over all observables as the gradient for a particular symbol)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "3WFJfFEbNT_-"
      },
      "outputs": [],
      "source": [
        "sum_of_outputs = tfq.layers.Expectation(\n",
        "    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))\n",
        "\n",
        "sum_of_outputs(my_circuit,\n",
        "               operators=[pauli_x, pauli_z],\n",
        "               symbol_names=['alpha'],\n",
        "               symbol_values=[[test_value]])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "-ujQUu3WNUAB"
      },
      "source": [
        "Here you see the first entry is the expectation w.r.t Pauli X, and the second is the expectation w.r.t Pauli Z. Now when you take the gradient:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "jcAQa9l0NUAB"
      },
      "outputs": [],
      "source": [
        "test_value_tensor = tf.convert_to_tensor([[test_value]])\n",
        "\n",
        "with tf.GradientTape() as g:\n",
        "    g.watch(test_value_tensor)\n",
        "    outputs = sum_of_outputs(my_circuit,\n",
        "                             operators=[pauli_x, pauli_z],\n",
        "                             symbol_names=['alpha'],\n",
        "                             symbol_values=test_value_tensor)\n",
        "\n",
        "sum_of_gradients = g.gradient(outputs, test_value_tensor)\n",
        "\n",
        "print(my_grad(pauli_x, test_value) + my_grad(pauli_z, test_value))\n",
        "print(sum_of_gradients.numpy())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "-fZmbYGANUAE"
      },
      "source": [
        "Here you have verified that the sum of the gradients for each observable is indeed the gradient of $\\alpha$. This behavior is supported by all TensorFlow Quantum differentiators and plays a crucial role in the compatibility with the rest of TensorFlow."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "lZsGG7lWNUAF"
      },
      "source": [
        "## 4. Advanced usage\n",
        "Here you will learn how to define your own custom differentiation routines for quantum circuits.\n",
        "All differentiators that exist inside of TensorFlow Quantum subclass `tfq.differentiators.Differentiator`. A differentiator must implement `differentiate_analytic` and `differentiate_sampled`.\n",
        "\n",
        "The following uses TensorFlow Quantum constructs to implement the closed form solution from the first part of this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "5iY4q6FKNUAG"
      },
      "outputs": [],
      "source": [
        "class MyDifferentiator(tfq.differentiators.Differentiator):\n",
        "    \"\"\"A Toy differentiator for <Y^alpha | X |Y^alpha>.\"\"\"\n",
        "\n",
        "    def __init__(self):\n",
        "        pass\n",
        "\n",
        "    @tf.function\n",
        "    def get_gradient_circuits(self, programs, symbol_names, symbol_values):\n",
        "        \"\"\"Return circuits to compute gradients for given forward pass circuits.\n",
        "        \n",
        "        When implementing a gradient, it is often useful to describe the\n",
        "        intermediate computations in terms of transformed versions of the input\n",
        "        circuits. The details are beyond the scope of this tutorial, but interested\n",
        "        users should check out the differentiator implementations in the TFQ library\n",
        "        for examples.\n",
        "        \"\"\"\n",
        "        raise NotImplementedError(\n",
        "            \"Gradient circuits are not implemented in this tutorial.\")\n",
        "\n",
        "    @tf.function\n",
        "    def _compute_gradient(self, symbol_values):\n",
        "        \"\"\"Compute the gradient based on symbol_values.\"\"\"\n",
        "\n",
        "        # f(x) = sin(pi * x)\n",
        "        # f'(x) = pi * cos(pi * x)\n",
        "        return tf.cast(tf.cos(symbol_values * np.pi) * np.pi, tf.float32)\n",
        "\n",
        "    @tf.function\n",
        "    def differentiate_analytic(self, programs, symbol_names, symbol_values,\n",
        "                               pauli_sums, forward_pass_vals, grad):\n",
        "        \"\"\"Specify how to differentiate a circuit with analytical expectation.\n",
        "\n",
        "        This is called at graph runtime by TensorFlow. `differentiate_analytic`\n",
        "        should calculate the gradient of a batch of circuits and return it\n",
        "        formatted as indicated below. See\n",
        "        `tfq.differentiators.ForwardDifference` for an example.\n",
        "\n",
        "        Args:\n",
        "            programs: `tf.Tensor` of strings with shape [batch_size] containing\n",
        "                the string representations of the circuits to be executed.\n",
        "            symbol_names: `tf.Tensor` of strings with shape [n_params], which\n",
        "                is used to specify the order in which the values in\n",
        "                `symbol_values` should be placed inside of the circuits in\n",
        "                `programs`.\n",
        "            symbol_values: `tf.Tensor` of real numbers with shape\n",
        "                [batch_size, n_params] specifying parameter values to resolve\n",
        "                into the circuits specified by programs, following the ordering\n",
        "                dictated by `symbol_names`.\n",
        "            pauli_sums: `tf.Tensor` of strings with shape [batch_size, n_ops]\n",
        "                containing the string representation of the operators that will\n",
        "                be used on all of the circuits in the expectation calculations.\n",
        "            forward_pass_vals: `tf.Tensor` of real numbers with shape\n",
        "                [batch_size, n_ops] containing the output of the forward pass\n",
        "                through the op you are differentiating.\n",
        "            grad: `tf.Tensor` of real numbers with shape [batch_size, n_ops]\n",
        "                representing the gradient backpropagated to the output of the\n",
        "                op you are differentiating through.\n",
        "\n",
        "        Returns:\n",
        "            A `tf.Tensor` with the same shape as `symbol_values` representing\n",
        "            the gradient backpropagated to the `symbol_values` input of the op\n",
        "            you are differentiating through.\n",
        "        \"\"\"\n",
        "\n",
        "        # Computing gradients just based off of symbol_values.\n",
        "        return self._compute_gradient(symbol_values) * grad\n",
        "\n",
        "    @tf.function\n",
        "    def differentiate_sampled(self, programs, symbol_names, symbol_values,\n",
        "                              pauli_sums, num_samples, forward_pass_vals, grad):\n",
        "        \"\"\"Specify how to differentiate a circuit with sampled expectation.\n",
        "\n",
        "        This is called at graph runtime by TensorFlow. `differentiate_sampled`\n",
        "        should calculate the gradient of a batch of circuits and return it\n",
        "        formatted as indicated below. See\n",
        "        `tfq.differentiators.ForwardDifference` for an example.\n",
        "\n",
        "        Args:\n",
        "            programs: `tf.Tensor` of strings with shape [batch_size] containing\n",
        "                the string representations of the circuits to be executed.\n",
        "            symbol_names: `tf.Tensor` of strings with shape [n_params], which\n",
        "                is used to specify the order in which the values in\n",
        "                `symbol_values` should be placed inside of the circuits in\n",
        "                `programs`.\n",
        "            symbol_values: `tf.Tensor` of real numbers with shape\n",
        "                [batch_size, n_params] specifying parameter values to resolve\n",
        "                into the circuits specified by programs, following the ordering\n",
        "                dictated by `symbol_names`.\n",
        "            pauli_sums: `tf.Tensor` of strings with shape [batch_size, n_ops]\n",
        "                containing the string representation of the operators that will\n",
        "                be used on all of the circuits in the expectation calculations.\n",
        "            num_samples: `tf.Tensor` of positive integers representing the\n",
        "                number of samples per term in each term of pauli_sums used\n",
        "                during the forward pass.\n",
        "            forward_pass_vals: `tf.Tensor` of real numbers with shape\n",
        "                [batch_size, n_ops] containing the output of the forward pass\n",
        "                through the op you are differentiating.\n",
        "            grad: `tf.Tensor` of real numbers with shape [batch_size, n_ops]\n",
        "                representing the gradient backpropagated to the output of the\n",
        "                op you are differentiating through.\n",
        "\n",
        "        Returns:\n",
        "            A `tf.Tensor` with the same shape as `symbol_values` representing\n",
        "            the gradient backpropagated to the `symbol_values` input of the op\n",
        "            you are differentiating through.\n",
        "        \"\"\"\n",
        "        return self._compute_gradient(symbol_values) * grad"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "bvEgw2m6NUAI"
      },
      "source": [
        "This new differentiator can now be used with existing `tfq.layer` objects:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "QrKnkWswNUAJ"
      },
      "outputs": [],
      "source": [
        "custom_dif = MyDifferentiator()\n",
        "custom_grad_expectation = tfq.layers.Expectation(differentiator=custom_dif)\n",
        "\n",
        "# Now let's get the gradients with finite diff.\n",
        "with tf.GradientTape() as g:\n",
        "    g.watch(values_tensor)\n",
        "    exact_outputs = expectation_calculation(my_circuit,\n",
        "                                            operators=[pauli_x],\n",
        "                                            symbol_names=['alpha'],\n",
        "                                            symbol_values=values_tensor)\n",
        "\n",
        "analytic_finite_diff_gradients = g.gradient(exact_outputs, values_tensor)\n",
        "\n",
        "# Now let's get the gradients with custom diff.\n",
        "with tf.GradientTape() as g:\n",
        "    g.watch(values_tensor)\n",
        "    my_outputs = custom_grad_expectation(my_circuit,\n",
        "                                         operators=[pauli_x],\n",
        "                                         symbol_names=['alpha'],\n",
        "                                         symbol_values=values_tensor)\n",
        "\n",
        "my_gradients = g.gradient(my_outputs, values_tensor)\n",
        "\n",
        "plt.subplot(1, 2, 1)\n",
        "plt.title('Exact Gradient')\n",
        "plt.plot(input_points, analytic_finite_diff_gradients.numpy())\n",
        "plt.xlabel('x')\n",
        "plt.ylabel('f(x)')\n",
        "plt.subplot(1, 2, 2)\n",
        "plt.title('My Gradient')\n",
        "plt.plot(input_points, my_gradients.numpy())\n",
        "plt.xlabel('x')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "oXqcJWigNUAL"
      },
      "source": [
        "This new differentiator can now be used to generate differentiable ops.\n",
        "\n",
        "Key Point: A differentiator that has been previously attached to an op must be refreshed before attaching to a new op, because a differentiator may only be attached to one op at a time."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "F_WHcj3bNUAM"
      },
      "outputs": [],
      "source": [
        "# Create a noisy sample based expectation op.\n",
        "expectation_sampled = tfq.get_sampled_expectation_op(\n",
        "    cirq.DensityMatrixSimulator(noise=cirq.depolarize(0.01)))\n",
        "\n",
        "# Make it differentiable with your differentiator:\n",
        "# Remember to refresh the differentiator before attaching the new op\n",
        "custom_dif.refresh()\n",
        "differentiable_op = custom_dif.generate_differentiable_op(\n",
        "    sampled_op=expectation_sampled)\n",
        "\n",
        "# Prep op inputs.\n",
        "circuit_tensor = tfq.convert_to_tensor([my_circuit])\n",
        "op_tensor = tfq.convert_to_tensor([[pauli_x]])\n",
        "single_value = tf.convert_to_tensor([[my_alpha]])\n",
        "num_samples_tensor = tf.convert_to_tensor([[1000]])\n",
        "\n",
        "with tf.GradientTape() as g:\n",
        "    g.watch(single_value)\n",
        "    forward_output = differentiable_op(circuit_tensor, ['alpha'], single_value,\n",
        "                                       op_tensor, num_samples_tensor)\n",
        "\n",
        "my_gradients = g.gradient(forward_output, single_value)\n",
        "\n",
        "print('---TFQ---')\n",
        "print('Foward:  ', forward_output.numpy())\n",
        "print('Gradient:', my_gradients.numpy())\n",
        "print('---Original---')\n",
        "print('Forward: ', my_expectation(pauli_x, my_alpha))\n",
        "print('Gradient:', my_grad(pauli_x, my_alpha))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "OGWcpqzDNUAP"
      },
      "source": [
        "Success: Now you can use all the differentiators that TensorFlow Quantum has to offer—and define your own."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "gradients.ipynb",
      "private_outputs": true,
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
